Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(shared-data, robot-server, api): Pipette configuration architecture refactor to organize by nozzle map and tip type #15250

Merged
merged 32 commits into from
Jun 11, 2024

Conversation

CaseyBatten
Copy link
Contributor

@CaseyBatten CaseyBatten commented May 22, 2024

Overview

Covers PLAT-334, with effects on PLAT-270, PLAT-269
Addresses concerns with the capability of pipette definitions to reflect the reality of pipette hardware performance and necessary configurations for any given nozzle layout/tip type combination.

A follow up PR will be expected to utilize the schema structure introduced here to implement new values for various pipettes and configurations.

Test Plan

Testing plan will involve ensuring migration of pipette definitions had maintained expected performance (speed of operation, pick up tip success) from previous infrastructure.

In terms of capability, the following protocols should pass analysis and execute successfully on a Flex and OT2.

  • Flex 1ch and 8ch protocol:
from opentrons import protocol_api
requirements = {
	"robotType": "Flex",
	"apiLevel": "2.18"
}

def run(ctx: protocol_api.ProtocolContext):
	tips = ctx.load_labware('opentrons_flex_96_tiprack_50ul', 'B3')
	tips2 = ctx.load_labware('opentrons_flex_96_tiprack_50ul', 'C3')
	multi = ctx.load_instrument('flex_8channel_50', 'right', tip_racks=[tips, tips2])
	single = ctx.load_instrument('flex_1channel_1000', 'left', tip_racks=[tips, tips2])
	labware = ctx.load_labware('nest_96_wellplate_100ul_pcr_full_skirt', 'D2')
	trash = ctx.load_trash_bin("D1")

	single.pick_up_tip()
	single.aspirate(10, labware.wells_by_name()["A1"])
	single.drop_tip()
	single.pick_up_tip()
	single.aspirate(10, labware.wells_by_name()["B1"])
	single.drop_tip()

	for i in range(11):
		multi.pick_up_tip()
		multi.aspirate(10, labware.wells()[i*8])
		multi.drop_tip()
		
	for i in range(12):
		multi.pick_up_tip()
		multi.aspirate(10, labware.wells()[i*8])
		multi.drop_tip()
  • Flex 96ch Protocol:
from opentrons.protocol_api import COLUMN, ALL

requirements = {
	"robotType": "Flex",
	"apiLevel": "2.18"
}

def run(protocol_context):
    adapter = protocol_context.load_adapter("opentrons_flex_96_tiprack_adapter", "B2")
    tip_rack1 = adapter.load_labware("opentrons_flex_96_tiprack_50ul", "B2")
    tip_rack2 = protocol_context.load_labware("opentrons_flex_96_tiprack_50ul", "C2")
    labware = protocol_context.load_labware('nest_96_wellplate_100ul_pcr_full_skirt', 'D2')
    trash = protocol_context.load_trash_bin('B3')
    instrument = protocol_context.load_instrument('flex_96channel_1000', mount="left", tip_racks=[tip_rack1])

    instrument.pick_up_tip()
    instrument.aspirate(10, labware.wells()[0])
    instrument.drop_tip()
        
    for i in range(6):
        instrument.configure_nozzle_layout(style=COLUMN, start="A1", tip_racks=[tip_rack2])
        instrument.pick_up_tip()
        instrument.aspirate(10, labware.wells()[0])
        instrument.drop_tip()

        instrument.configure_nozzle_layout(style=COLUMN, start="A12", tip_racks=[tip_rack2])
        instrument.pick_up_tip()
        instrument.aspirate(10, labware.wells()[0])
        instrument.drop_tip()
  • OT2 1ch and 8ch protocol:
from opentrons import protocol_api
requirements = {"robotType": "OT-2", "apiLevel": "2.17"}

def run(ctx: protocol_api.ProtocolContext):
    tips = ctx.load_labware("opentrons_96_tiprack_300ul", 4)
    tips2 = ctx.load_labware("opentrons_96_tiprack_300ul", 1)
    plate = ctx.load_labware("nest_96_wellplate_200ul_flat", 2)
    multi = ctx.load_instrument('p300_multi_gen2', 'right', tip_racks=[tips, tips2])
    single = ctx.load_instrument('p300_single_gen2', 'left', tip_racks=[tips, tips2])

    single.pick_up_tip()
    single.aspirate(10, plate.wells_by_name()["A1"])
    single.drop_tip()
    single.pick_up_tip()
    single.aspirate(10, plate.wells_by_name()["A2"])
    single.drop_tip()

    for i in range(11):
        multi.pick_up_tip()
        multi.aspirate(10, plate.wells()[i*8])
        multi.drop_tip()

    for i in range(12):
        multi.pick_up_tip()
        multi.aspirate(10, plate.wells()[i*8])
        multi.drop_tip()

Changelog

  • Introduced validNozzleMaps definition within pipette definition, detailing layout of hardware-testing validated nozzle maps.
  • Refactored currentByTipCount, speedByTipCount and distanceByTipCount to instead be configuration value members of a double keyed schema utilizing a nozzle layout map key and tip type.
  • Relocated tip overlap dictionaries into the configuration value structures within the configurationByNozzleLayout category of the pipette definitions
  • Refactored engine and robot-server logic to utilize configurationByNozzleLayout in place of the current/speed/distance by tip type methodology
  • Refactored JS types/tests and Mutable configurations to account for new pipette schema shape

Review requests

Please take a look at the pipette definitions schema and the migrated definitions for the pipettes. Does this seem appropriate and comprehensive enough to capture the data we are seeking to include in our system? Does the engine side implementation of this new approach seem appropriate?

Risk assessment

Medium - While most of these changes are to the pipette schema and non-invasive to engine logic, I would like a careful eye/testing regiment for the various pipette definitions that have been migrated to the new schema type to ensure performance remains the same without regressions.

Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick glance, looks like the right path but there's some stuff I'd like to see changed

def _get_matching_approved_nozzle_map(self) -> str:
for map_key in self._valid_nozzle_maps.maps.keys():
if self._valid_nozzle_maps.maps[map_key] == list(
self._nozzle_manager.current_configuration.map_store.keys()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a whole lot to be doing just here, and it adds this error category. instead of doing this, let's tell the nozzle manager what map key it's using and then we can look up the configuration based on that directly.

self._working_volume = min(tip_type.value, self.liquid_class.max_volume)

def _get_matching_approved_nozzle_map(self) -> str:
for map_key in self._valid_nozzle_maps.maps.keys():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see the ot2 comment

approved_map = self._get_matching_approved_nozzle_map()
try:
if isinstance(config, PressFitPickUpTipConfiguration) and all(
[
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe simplify this by putting config.configuration_by_nozzle_map[approved_map] in an intermediate

):
approved_map = map_key
if approved_map is None:
raise ValueError(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be a protocol engine error that descends from an enumerated error

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still need this one

return config.configuration_by_nozzle_map[
self._nozzle_manager.current_configuration.valid_map_key
][pip_types.PipetteTipType(self._liquid_class.max_volume).name].speed
except KeyError:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should have these errors be enumerated errors defined in shared data using an error code, like a new 4000-series "missing configuration data" or something

self._nozzle_manager.current_configuration.valid_map_key
][pip_types.PipetteTipType(self._liquid_class.max_volume).name].speed
except KeyError:
default = config.configuration_by_nozzle_map[
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be a free function in a shared area? I think it's an identical implementation for the two machines

except KeyError:
default = config.configuration_by_nozzle_map[
self._nozzle_manager.current_configuration.valid_map_key
].get("default")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and actually generally the task of "get data from the unified structure" might be able to be commonized further

Copy link

codecov bot commented Jun 4, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.18%. Comparing base (7de5e83) to head (8f4040b).
Report is 82 commits behind head on edge.

Current head 8f4040b differs from pull request most recent head d33c225

Please upload reports for the commit d33c225 to get more accurate results.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             edge   #15250      +/-   ##
==========================================
- Coverage   63.20%   63.18%   -0.02%     
==========================================
  Files         287      287              
  Lines       14891    14875      -16     
==========================================
- Hits         9412     9399      -13     
+ Misses       5479     5476       -3     
Flag Coverage Δ
g-code-testing 92.43% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...-data/python/opentrons_shared_data/errors/codes.py 93.82% <ø> (ø)
.../python/opentrons_shared_data/pipette/load_data.py 90.90% <ø> (ø)
...n/opentrons_shared_data/pipette/model_constants.py 100.00% <ø> (ø)
...rons_shared_data/pipette/mutable_configurations.py 87.64% <ø> (+0.93%) ⬆️
...pentrons_shared_data/pipette/pipette_definition.py 94.17% <ø> (-0.18%) ⬇️
...s_shared_data/pipette/scripts/build_json_script.py 0.00% <ø> (ø)

@CaseyBatten CaseyBatten marked this pull request as ready for review June 6, 2024 15:02
@CaseyBatten CaseyBatten requested a review from a team as a code owner June 6, 2024 15:02
@CaseyBatten CaseyBatten requested review from a team as code owners June 6, 2024 15:02
@CaseyBatten CaseyBatten requested review from b-cooper and nusrat813 and removed request for a team June 6, 2024 15:02


class MissingConfigurationData(GeneralError):
"""An error indicating that provided configurationd ata is missing or invalid.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""An error indicating that provided configurationd ata is missing or invalid.
"""An error indicating that provided configuration data is missing or invalid.

Copy link
Contributor

@b-cooper b-cooper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Talked through in person and collected a list of comments offline. After those are addressed and Seth's comments too, this looks good to merge!

Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it will work, and good job - this was pretty hairy! Would love to see the virtual static pipette checks raise enumerated errors and I think there's a solid followup that takes some of these functions that have like 3 layers of try/except logic and factors them into using some intermediate accessors, though.

@@ -280,9 +284,20 @@ def build(
f"Partial Nozzle Layouts may not be configured to contain more than {MAXIMUM_NOZZLE_COUNT} channels."
)

validated_map_key = None
for map_key in valid_nozzle_maps.maps.keys():
if valid_nozzle_maps.maps[map_key] == list(map_store.keys()):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are these sorted explicitly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validated maps are ordered from the smallest validated configuration to the largest in our definitions, with nozzle sets in each map following a Column then Row mapping order, the same as our map store.

):
approved_map = map_key
if approved_map is None:
raise ValueError(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still need this one

@CaseyBatten CaseyBatten merged commit b14985c into edge Jun 11, 2024
41 of 42 checks passed
@CaseyBatten CaseyBatten deleted the pipette_configuration_architecture_by_nozzle_map branch June 11, 2024 19:10
aaron-kulkarni pushed a commit that referenced this pull request Jun 13, 2024
…tecture refactor to organize by nozzle map and tip type (#15250)

Covers PLAT-334, with effects on PLAT-270, PLAT-269
Addresses capability for nozzle layout and tip type to effect current, speed, distance and tip overlap
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants